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ABSTRACT 

We compare potential state-of-the-art experiments for detecting Earth-mass planets around main- 
sequence stars using radial velocities, transits, astrometry, and microlensing. For conventionally- 
discussed signal-to- noise ratio (S/N) thresholds, S/N ~ 8, the last three methods are roughly compa- 
rable in terms of both the total number of planets detected and the mass distribution of their host 
stars. However we argue that S/N ~ 25 is a more conservative and realistic S/N threshold. We show 
analytically and numerically that the decline in the number of detections as a function of S/N is very 
steep for radial velocities, transits, and astrometry, such that the number of expected detections at 
S/N ~ 25 is more than an order-of-magnitude smaller than at conventional S/N thresholds. Indeed, 
unless Earth-mass planets are very common or are packed much closer to their parent stars than in 
the solar system, future searches using these methods (as they are currently planned) may not yield 
any reliable Earth-mass planet detections. On the other hand, microlensing has a much shallower 
S/N slope than the other techniques and so has much greater sensitivity at realistic S/N thresholds. 
We show that even if all stars have Earth-mass planets at periods of one year (and adopting other 
optimistic assumptions as well), the combined yield of all four techniques would be the detection of 
only about five such planets at S/N ~ 25. 

Subject headings: astrobiology - astrometry - planetary systems - extraterrestrial intelligence 



1. INTRODUCTION 

The detection of terrestrial planets around main- 
sequence stars represents a long-standing major goal of 
observational astronomy that may finally be realized 
within the next decade. Terrestrial (i.e. rocky) plan- 
ets are interesting for several reasons. The frequency, 
mass, and orbital distribution of planets with masses of 
'--^ 1 — 10 Earth masses (Mg) yield strong constraints 
on the physical mechanisms at work in planetary forma- 
tion and would provide a crucial observational test of 
planet formation theories. When combined with similar 
distributions of the properties of more massive planets, 
this information would allow for stringent tests of core- 
accretion m odels fo r the formation of gas and ice-giant 
planets fe.g. lIda~L in 2004). Terrestrial planets are also 
commonly assumed to be the most favorable places to 
look for extrasolar life. Especially favored are planets 
with separations from their parent star that place them 
in the so-called "habitable zone," roughly the range of 
distances at which liquid water can exist, although it is 
important to keep in mind that not all low-mass planets 
in the habitable zone are ne cessarily terrestrial ( Kuchncr 
|2003';"Ravmond et al. 2004), nor should gas-giant planets 
in the habitable zone be disregarded, as their satellites 
may well be capable of supporting hfe l|Williams et al.l 
IT997I) . 

An accurate and robust determination of ry^, the 
frequency of Earth-mass planets around main-sequence 
stars, is also essential for the design of future missions 
that aim to directly image and take spectra of terres- 
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trial planets around the nearest stars, such as the Ter- 
restrial Planet Finder^ and Darwin^. In many design 
concepts for these missions, the required diameter D of 
the primary collecting mirror is directly prop ortional to 
the d istance d of the stars to be surveyed ijBeichmarJ 
l2004j) . Thus for a fixed number of planets in the survey 

— 1/3 

volume, D oi r/^ ' . Since the cost and feasibility of such 
missions is likely to be an extremely strong function of 
D, ac curate determination of 77^ is essential ( BeichmanI 
1200^ . 

Several different indirect detection techniques have the 
potential to determine the frequency of terrestrial plan- 
ets around solar-type stars. Transiting terrestrial plan- 
ets are detectable photometrically via the periodic dim- 
ming of their parent star. For a planet with the radius 
of the Earth orbiting a main-sequence star, the frac- 
tional change in brightness during the transit is O(10~*); 
achieving photometric precisions at this level or bet- 
ter generally re quires observations from space (but see 
iGould et al J 1200 3b). Also, the low transit probability 
and duty cycle requires that many stars must be moni- 
tored continuously. The space mission Kepler^ will con- 
tinuously monitor ~ 10^ stars for four years in order 
to search for transiting planets and will be sensitive to 
terrestrial planets orbiting in the habitable zones of disk 
main-sequence stars at distances of d ~ 1 kpc. Terrestrial 
planets can also be detected from the astrometric wob- 
ble they induce on their parent star. For planets of mass 
~ A/q orbiting at a few AU from parent stars located at 
distances of a few parsecs, the induced displacement is 
0(/ias). The Space Interferometry Mission (SIMJ^ will 
use ultra-high precision astrometry to search for planets 

* http:/ /planetquest.jpl. nasa.gov/TPF/tpf_index. html 
^ http:/ /www.esa.int/science/darwin 
^ http://www.kepler.arc.nasa.gov/ 
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in the solar neighborhood {d < 20 pc), and wiU be sen- 
sitive to planets of a few Earth-masse s with orbits of a 
few AU around the most nearby stars llFord fc Tremaind 
1^03; Gould ct al. 20032 iSozzetti et al.ll2002|) . Terres- 
trial planets can be detected via microlensing from the 
pronounced but brief deviation the y induce on microlens- 
ing events of their pa rent stars l|Gould fc Loebl 119921: 
iMao fc Paczynskil991j) . Microlensing is potentially very 
sensitive to Earth-mass planets (Bennett & Rhic 1996). 
However, the rarity and brevity of the signal induced by 
the planets requires the continuous monitoring of a large 
number of stars. The prop osed space mission Mic rolens- 
ing Planet Finder (MPF) (jBennett fc R,hiell2002D would 
search for planets via microlensing by continuously mon- 
itoring ~ 10® main-sequence stars toward the Galactic 
bulge. This mission would be sensitive to terrestrial plan- 
ets with few AU orbits around Galactic disk and bulge 
main-sequence stars at distances of 1 — 10 kpc. Terrestrial 
planets ma y also be detectable from the groun d with mi- 
crolensing (|Bennettll20'0l iGaudi et al]l2004D . Orbiting 
planets induce radial velocity (RV) variations on their 
parent stars: an Earth-mass planet orbiting a solar- type 
star with a period of 0.5 yr induces a RV variation on 
its parent star of amplitude ^ 0.1 m s^^. This is well 
below the systematic limit of 1 m s~^ typically quoted 
for current RV surveys. However, it is not clear whether 
this represents a hard floor, or whether the systematic 
errors can be averaged out over repeated observations. 
Therefore, it may also be possible to detect Earth-mass 
planets from the ground with RV. 

Although extrasolar Earth-mass planets have yet to 
be discovered orbiting main-sequence stars, they have 
in fact already been dis covered a round a neutron 
star using pulsar timin g feonacki fc Wolsz czaiil 120031: 
iWolszczan fc Fraill [1991 iWolszczanI 11994^) . However, 
subsequent pulsar searches have shown pulsar planets to 
be a very rare phenomenon. Furthermore, such planets 
proba bly formed after the creation of the pulsar (,Hanser3 
l2002j) and are unlikely to be suitable for life. 

All of the proposed detection methods discussed above 
are difficult and require ambitious, cutting-edge exper- 
iments. They are therefore both quite expensive and 
complicated, and thus they require very careful and con- 
servative justification that is based not only on their pro- 
posed scientific return, but also on their feasibility and 
probability of success. To date, discussion has therefore 
focused on two key issues. First, what information will 
be learned about such planets if they are detected? Sec- 
ond, how many planets can be detected down to a given 
threshold in signal-to- noise ratio S/N? However, there 
is a third dimension to this discussion that has mostly 
been ignored: what is the expected distribution of S/N 
with which planets are detected? 

At first sight, this appears to be a rather arcane ques- 
tion. In fact, it is crucial. We will argue that, in sharp 
contrast to experiments that can easily be repeated or 
that return null results, difficult-to-repeat astronomical 
experiments that purport to detect novel phenomena 
must have a high S/N threshold to be believable. Or, 
what amounts effectively to the same thing, a substantial 
fraction of the detections must be at high S/N, thereby 
allowing one to characterize the believability of the detec- 
tions at lower S/N. This rule of thumb is unconsciously 
followed by all observers, or at least all observers who 



develop respectable track records over the long term. It 
should also be applied to the daunting task of character- 
izing Earth-mass planets. 

Here we quantify the sensitivity of future experiments 
to Earth-mass planets using the four proposed tech- 
niques: transits, astrometry, RV, and microlensing. We 
begin by discussing the complementarity of these four 
techniques, both in terms of the physical parameters 
measured and the orbital separations of the planets (321 ■ 
We then state the methods and assumptions used in eval- 
uating the sensitivity of these techniques (Q. We next 
discuss the expected form of the S/N distribution for 
each of the four techniques (2J. We present a number of 
arguments suggesting that conventional S/N thresholds 
are inadequate, and as a result that the number of reli- 
able detections is likely to be considerably smaller than 
currently envisioned (33- We then explore the number 
of expected planet detections from each technique (SjHJ as 
a function of the signal-to- noise threshold f H6.1|) . the pri- 
mary mass f ii6.2|) . planet period (' ^t).3|l . and planet mass 
f ti5.4|) . We conclude with a discussion of the implications 
of our results (iQ. 

2. COMPLEMENTARITY OF PROPOSED TECHNIQUES 

Different search techniques are intrinsically sensitive 
to terrestrial planets with different characteristics. In 
terms of orbital separation, astrometric sensitivity scales 
as the planet semi-major axis, a, while for RV, the sensi- 
tivity is oc a~^/^. The sensitivity of a magnitude-limited 
transit survey is oc a~^/^, but this technique is partic- 
ularly prized for its ability to detect terrestrial plan- 
ets in the habitable zone. Microlensing is most sensi- 
tive to planets near the Einstein ring of the primary 
star, which for typical lensing geometries lies at about 
a ^ 2.5 AU. These methods are also complementary in 
the sense that they measure different physical proper- 
ties of the planets. Transits provide planet radii, while 
astrometric measurements provide planet masses. Ra- 
dial velocity measurements yield the mass of the planet 
modulo the sine of the inclination of the planetary or- 
bit. In its primitive incarnation, microlensing routinely 
measures only the mass ratio between the planet and 
star, although for ambitious next-generation surveys, the 
mass of the planet w ill be measurable for a large frac- 
tion of the detections ("Benne tt fc RMel2002HGould et alJ 
^2003: Han ct al. 2004). Each method also has other 
unique advantages. For example, only astrometry and 
RV routinely probe the architecture of multiple-planet 
systems, whereas only microlensing can detect Earth- 
mass planets at very large radii, or free-floating Earth- 
mass planets that hav e been ejected fro m their systems 
ifBennett fc Rhic 200l iHan etaI]l2?iol . 

These differences in sensitivity are basically a good 
thing: they imply that a coordinated attack on this dif- 
ficult problem from several directions can substantially 
increase the range of planet parameters and properties 
probed. However, while beneficial to science, these dif- 
ferences make it difficult to directly compare the over- 
all sensitivity of different techniques to the detection of 
terrestrial planets. Fortunately, the main result of this 
paper, a comparison of the relative distribution functions 
of S/N, is generally unaffected by exactly how the com- 
parison is carried out. Rather, this affects only the nor- 
malizations of these distribution functions. 
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3. METHODS AND ASSUMPTIONS 

In this section we provide our methods and assump- 
tions used in calculating the number of terrestrial plan- 
ets detectable by each of the four techniques considered. 
To compute the number of detectable planets, we must 
specify both the frequency and properties of the plan- 
ets, as well as the the experimental parameters for each 
technique. 

For most of the results presented in this paper, we 
consider only planets of mass M = (or radius 

R = i?©). This is because all the currently known terres- 
trial planets* have M < M©. However, current theories 
of planet formation predict that terrestrial planets may 
have masses of up to ~ 10 M©, and we therefore consider 
the sensitivity of the techniques as a function of planet 
mass in ti6.4l In order to be as democratic as possible, 
we begin by assuming orbital periods that allow each 
method to play to its strengths. For astrometry, RV, and 
microlensing, we assume that the planets are all located 
at periods at which the detection sensitivity of the par- 
ticular method is maximized (or nearly so). For transits, 
we assume the planets are located in the habitable zones 
of their parent stars. Our specific choices are somewhat 
arbitrary, and in i)6.3l we explore the sensitivity of the 
various methods as a function of orbital period. Finally, 
we assume that every main-sequence star surveyed has a 
planet of the specified parameters. 

We specify the experimental parameters for each tech- 
nique that are appropriate to either planned space mis- 
sions, or future ground-based surveys of our own con- 
ception. Again, our choices here are somewhat arbitrary, 
and one could consider other choices for the experimental 
parameters. However, we consider our choices to repre- 
sent the most ambitious yet feasible experiments of their 
kind, and therefore to be very optimistic. 

Radial Velocity: We assume that each star has a planet 
with M = Mq) and period P = 0.5 yr. We assume a 
total observing time and photon collection rate appro- 
priate to four dedicated 2m telescopes over 5 years. This 
corresponds to 60,000 hours of dedicated observations. 
We assume a systematics limit of cr = Ims^^ and take 
the error to be cr = (1 -h loO-4[^-i-5])i/2 mg-i for a 2 
minute exposure. Note that, in terms total photon col- 
lecting power, this is approximately equivalent to 500 
hours per year for 5 years on Keck. However, the combi- 
nation of four dedicated 2m telescopes is much superior 
to the Keck scenario for stars V < 5 because the fixed 
overhead time per target is a much smaller percentage of 
the exposure time. 

Astrometry: We assume that each star has a planet 
with M = M© and P = 3yr. We assume a survey of 
40,000 astrometric measurements each with systematics- 
limited errors of cr = 1 /xas. This corresponds to 2,000 
hours of SIM time per year over 5 years, with each 
measurement taking 15 minutes. To account for the 
photon noise of faint stars, we assume an error a ~ 

(l + 100-4[^-"l)i/Vs. 

Transits: We assume each star has a planet with R — 
i?0 located in the habitable zone, defined as a = 

* Around main-sequence stars. 



(Lboi/iboi,0)^/^AU, where Lboi/^boi,© is the bolomet- 
ric luminosity of the parent star relative to the Sun. We 
use the parameters of the Kepler satellite, except that 
we augment t he sample to include faint nearby stars as 
advocated bv iGould et al.l ((2003b), thereby multiplying 
the detection rate by a factor 2.5 relative to the original 
Kepler strategy. 

Microlensing: We assume that each main-sequence star 
has an Earth-mass planet at a = 2.5 AU. We use the 
results of a simulation by iGaudi et alJ l|2004f l of a 5- 
year ground-based microlensing search using four 2m 
wide-field telescopes located in Chile, South Africa, Aus- 
tralia, and Hawaii. This simulation adopts the star- 
count b ased norma lization of the optical depth calcu- 
lated bv lHan fc G~ uld (2003). The form of the derived 
S/N distribution is similar to that of the MPF satellite 
(jBennett & Rhie 2002), but the normalization is about 
30% smaller when the latter is rescaled using the same 
optical depth derived bv lHan fc Gouldl ()2003|) . 

For transits and microlensing, the above descriptions 
serve to completely describe the experiment, but for as- 
trometry and RV they do not. Transits and microlens- 
ing are carried out in survey mode, so that every star 
is, by definition, observed in exactly the same way. As- 
trometry and RV are pointed experiments in which the 
observer is free to divide up the observing time among 
targets with more or less complete freedom. In our base- 
line comparison, we will force astrometry and RV into 
survey mode. That is, we will demand that they allocate 
the same amount of observing time to each of their pro- 
gram stars. We determine the number of program stars, 
by rank ordering all stars in the Hipparcos catalog 
(|ESAII1997f) according to their sensitivity under each re- 
spective method, and then requiring that a 1 M© planet 
in a favorable orbit (edge-on for RV, face on for astrome- 
try) around the A^th star can be detected at S/N = 5 by 
using 1/A^ of the total observing time. This may not be 
the optimal choice and is certainly not how current RV 
surveys are carried out. Hence, in S 14. II we also consider 
alternative observing scenarios. 

Of course, our assumptions are idealized in that they 
do not take account of both recognized and unrecognized 
effects. For example, photospheric variations will prevent 
a = lms~^ measurements for some RV targets; some 
reference stars required to fix the local reference frame 
of astrometric targets will be found to be astrometrically 
unstable; some transit targets will be too variable for 
reliable photometric monitoring; we ignore binary stars 
completely. Since problems of this type are likely to af- 
fect all methods, and their actual magnitude cannot be 
estimated with high precision in advance, we ignore them 
here. As nearly all unaccounted-for effects will serve to 
degrade the sensitivities, our assumptions tend to be op- 
timistic. 

4. SCALING WITH SIGNAL-TO-NOISE RATIO 

Figure H and Table 1 compare the distribution of 
signal-to-noise ratios of all four methods. Here we de- 
fine the signal-to-noise ratio as S/N = (Ax^)^/^, where 
Ax^ is the difference in in the best-fitting models with 
and without a planet. The scalings with S/N are all ba- 
sically power laws N cx (S/N)"^^^, except for the case 
of microlensing, for which the scaling is a broken power 
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Signal-to-Noise Ratio (S/N) 

Fig. 1. — Number of detectable Earth-mass planets as a func- 
tion of threshold in S/N. Each experiment is allowed to play to its 
strengths; it is assumed that all stars have planets with: P = 0.5 yr 
for the RV survey; P = Syr for astrometry (SIM); a = 2.5 AU for 
microlensing; and a = (I'bol/-^'bol q)^^^^^ for transits. Symbols 
show the actual S /N of detections in two challenging historical ex- 
periments. Circles passed the basic cuts as microlensing events to- 
ward the LMC as reported bvlAlcock et alj l200Cf) . but circles with 
crosses were later found to be background SNe, not microlensing 
events. Triangles are RV detections of planets. Filled triangles 
are detections that are ambiguous, controversial, or have been re- 
tracted. 



law. Why do the functions take this form and why is the 
slope for microlensing so much shallower than the other 
methods? While each curve is the result of a complex 
numerical integration, it is nevertheless possible to basi- 
cally understand the origins of all four slopes using fairly 
simple reasoning. In each case, we need to ask, given an 
ensemble of detections with S/N above some threshold, 
what subset of these detections will have twice that S/N? 

Let us begin with Kepler transits. Consider a star of 
given spectral type for which a planet can just barely 
be detected at S/N — (S/N)inin- Where can one find a 
star of the same spectral type with S/N = 2(S/N)min? 
Clearly, if another star were closer by a factor 2, it would 
yield 4 times greater flux and so twice as much S/N. 
However, there are 8 times fewer of such nearby stars, so 
v = 3. Why then is the slope in Figure [T]j/ = 2.4 and 
not exactly 3? As discussed by Gould et al. (2003b), the 
Kepler 10" point spread function (PSF) contains about 
y = 17 of sky, which fundamentally limits its sensitivity 
to otherwise detectable planets around M dwarfs. For 
each adopted threshold (S/N)min, there is some spectral 
class of stars for which coming closer by a factor 2 moves 
the star from below the sky to above it, and this implies 
that it is not quite necessary to reduce the sample size by 
a factor 8 to double (S/N)min. If all stars were above the 
sky, the slope would be = 3. If all were below the sky, 
the slope would be // = 1.5. The actual value oi v = 2.4 
results from competition between these two regimes. 

The reasoning is quite similar for SIM astrometry. 
In this case, the astrometric precision does not depend 



on distance for most stars that will be monitored be- 
cause the systematics limit sets in relatively faint at 
y = 11. Since the astrometric signature scales inversely 
as the distance, while the astrometric noise is distance- 
independent, the S/N also scales inversely as the dis- 
tance. So, just as with Kepler, one expects i/ = 3. How- 
ever, just as with Kepler, this scaling is broken by a 
second-order effect. Some stars are ^ ^ 11 and hence 
are in the photometric not systematics limit. In the pho- 
tometric limit, one expects v = 1.5. The net value of 
v — 2.4 again results from competition. 

For RV, the situation is qualitatively similar. In the 
photon limit, one expects v = 3, just as for Kepler. Al- 
though very few stars have V < 1.5 (the regime domi- 
nated by systematics), some are this bright. Once sys- 
tematics set in, there is no improvement in S /N obtained 
by moving the star closer. Hence, = oo. The net re- 
sult is that 1/ = 3.1, i.e., slightly higher than the value 
obtained in the photon limit. 

For microlensing, the situation is very different from 
the other three methods. First, the sources of light are 
independent of the planets and their hosts, and further- 
more, are all at essentially the same distance (i.e., the 
Galactocentric distance Rq ^ 8 kpc). Therefore, the 
distribution of photometric errors directly reflects the 
shape of the source-star luminosity function. Second, 
there is large distribution of signal amplitudes i5 for a 
given planetary system. Therefore, in order to find a 
higher S/N event, one can either look at a brighter source 
stars, or "wait" for the rare events that have higher in- 
trinsic signal amplitudes. It is possible to demonstrate 
numerically that, for a planet with mass ratio q located 
near the Einstein ring of its primary, the area of the 
Einstein ring that is photometrically perturbed by the 
planet by a fractional amount > 5 is fi oc qS~^. This 
is in the limit of point sources; in reality this distribu- 
tion will be cut off at a some maximum deviation (5max 
due to flnite-source effects. The luminosity function in 
the Galactic bulge is well-represented by a broken power- 
law, with dTV/dlogL oc L~^/^ for stars fainter than the 
turn-off, and oc L~^-^ f or stars brighter than the turn-off 
l)Holtzman et alJll998D . Stars fainter than the turn-off 
are fainter than the combined sky plus unresolved star 
background, and thus S/N cx Lil,^^^S oc Lq^^^S^^"^, where 
the ri^/^ term arises from the assumption that the typical 
duration of perturbations is oc Q^^^. Stars brighter than 
the turnoff are typically above the background, and thus 
S/N oc L^^-^q^^^S^^^ . These basic ingredients combine to 
yield the broken power-law behavior in seen in Figure ^ 
We note that, for a space-based mission such as MPF, 
essentially all sources are above the background. There- 
fore, for sources below the turn-off the scaling with S/N 
is expected to be steeper than for the ground-based sce- 
nario (although the overall normalizatio n is higher) , as is 
seen in the results of the simulations of lBennett fc R.hi3 
ll200l . 



4.1. Flexibility of Astrometry and RV 

Recall that in constructing Figure ^ we handicapped 
the astrometric and RV experiments to mimic surveys. 
That is, we required them to expend the same observing 
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Number of Detectable Planets 



Method 


S/N > 


5 


S/N > 


10 


S/N > 


25 


S/N > 


50 


S/N > 100 


RV (Survey) 


22 




2 

















SIM (Survey) 


207 




65 




7 




2 







Microlensing 


208 




121 




46 




26 




11 


Kepler 


246 




49 




6 




1 









S/N = 


5 


S/N = 


10 


S/N = 


25 


S/N = 


50 


S/N = 100 


RV (Optimized) 


38 




15 




5 




2 







SIM (Optimized) 


335 




174 




76 




37 




17 



time on all targets. The transit and microlensing surveys 
must do this, but in reality astrometry and RV are not 
so constrained. We illustrate this flexibility by consider- 
ing an alternative set of strategies. Here we first choose 
an arbitrary number TV and pick out the N stars with 
the best sensitivities from our rank-ordered list of Hip- 
parcos stars. We divide all the available observing time 
among these N stars in such a way that each has equal 
final S/N. That is, the worst stars from among the list 
are observed more frequently (or with longer exposures) 
than the best. The result of this exercise is shown by 
the chains of filled circles in Figure |21 which should be 
contrasted with the corresponding curves taken from Fig- 
ure n Table 1 compares the number of detected planets 
at or above several selected S/N rmder these two differ- 
ent observational strategies. It is important to emphasize 
that the curves represent one experiment, while the cir- 
cles represent many different possible experiments, only 
one of which can be realized. That is, if every star has a 
planet, and the astrometric observations follow the sur- 
vey strategy, then there will be three detections with 
S/N > 40 and 30 detections with S/N > 16. However, 
if the alternative strategy is used, one has a choice of 
three detections with S/N = 500 or 30 detections with 
S/N = 60. One cannot have both. 

5. CONSERVATIVE SIGNAL-TO-NOISE THRESHOLDS 

What is the appropriate S/N cutoff? The answer to 
this question depends on one's definition of "appropri- 
ate." Traditionally, the appropriate value of the min- 
imum S/N threshold has been set by just considering 
statistics. One can ask, given the expected properties 
of the noise and signal, what S/N threshold is neces- 
sary in order that less than some tolerated number of 
false alarms are expected. Determination of this "sta- 
tistical" (S/N)min can be quite comphcated and requires 
careful consideration of both the expected distribution of 
measurement noise as well as the number of statistically 
independ ent trials needed to thoroughly search the data 
(see, e.g.. lJenkins et al]l2002l) . 

Thresholds set based on statistics alone are the least 
conservative and the most optimistic, in the sense that it 
is statistically impossible to detect a signal with any con- 
fidence at a lower threshold. While such thresholds max- 
imize the number of detected planets, we argue that they 
are dangerous and undesirable for several reasons. First, 
for the majority of the "detections" , which are near this 
minimum threshold, one has used essentially all of the 
available information simply to detect the signal. This 
means that it is generally impossible to determine the na- 
ture of the signal, i.e., to measure planetary parameters 
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Fig. 2. — Flexibility of RV and astrometry (SIM). Curves are 
the same as Fig.lTland represent the sensitivity of these techniques 
if they are forced into "survey mode" , in which each target star is 
observed for the same total amount of time. Dashed curve is for 
RV, dotted curve is for SIM. Circles represent a series of different 
experiments, in each of which the exposure times are tailored to 
obtain the same S/N for each target star. That is, there are more 
detections at a fixed S/N but at the price of not having a tail of 
detections at substantially higher S/N. Filled circles are for SIM, 
open circles are for RV. 



or to use the detailed form of the signal to corroborate its 
interpretation. Simply speaking, all one is confident of 
is that something has been detected - there is essentially 
no information about exactly what has been detected. 
As a result, such detections are extremely prone to being 
confused with false positives. Given that, for many of 
these experiments, follow-up and independent confirma- 
tion of the candidates will be difficult or impossible, this 
is a serious concern. We discuss this in more detail be- 
low. Furthermore, even if false positives can be reliably 
excluded, it is still the case that the parameters of the 
detected planets will be very poorly constrained, making 
the scientific usefulness of the detections questionable. 

We argue that the actual S/N threshold adopted 
should consider both the ability to extract physical pa- 
rameters from the detected planets, as well as the possi- 
ble presence of false positives, and the inability to per- 
form follow-up of the detections. The threshold required 
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to measure planetary parameters to a given precision de- 
pends on many different quantities, such as the complex- 
ity of the signal, the cadence, sampling, and duration 
of the observations, etc. This issue ha s been most thor- 
ough l y explored in the case of SIM l|Ford fc Tremain'3 
l20mt ' lSc«zetti et al.ll2nnl r20n3). Very roughly, these 
authors find that thresholds of S/N > 25 are required 
to constrain the planetary parameters to better than 
30%. For transits, the situation is considerably bet- 
ter, as the period of transiting planets should be mea- 
sured with exquisite precision even for low S/N detec- 
tions, and the fractional error in the planetary radius is 
- 0.5(S/N)-i < 2% for S/N > 25. For microlensing, 
the simulations of iGaudi et alJ l)2004D indicate that, for 
90% of the perturbations detected with S/N > 25, the 
planet /star mass ratio should be measurable to < 30%. 
No comprehensive study of the ability to extract planet 
parameters from RV curves has been performed, but it 
seems likely that, given the nature of the observations, 
the uncertainties will be similar to those for astrometric 
data. 

What S/N threshold is required to deal with false pos- 
itives? Until the experiments are actually carried out, 
and false positives are identified, one cannot be certain. 
However, it is instructive to consider historical precedent. 
Figure n summarizes two relevant experiences. The first 
is the 5.7-year microlensing dark-matter search toward 
the Large Magellanic Cloud (L MC) by the MACHO col- 
laboration ( Alcock et alJ200nt) . Like the proposed planet 
searches, the MACHO experiment was a massive search 
for objects that are extremely difficult to detect. Because 
the detections involve events that are over, and hence can 
never be verified individually, MACHO was compelled to 
be very conservative in what they called a "detection". 
They demanded (S/N)min = 20. Nevertheless, there is 
substantial debate in the microlensing community as to 
whether they were conservative enough. Figure ^ illus- 
trates one reason why. The circles show the S/N of all the 
events surviving the basic selection cuts. However, circles 
with crosses were eliminated because they were judged 
to be probable supernovae (SNe). How they came to be 
recognized as such is very instructive in the current con- 
text. Originally, SNe were not considered as a possible 
background. It was only because two of the SNe "events" 
had very high S/N ^ 60 (see Fig.^ that their character 
as SNe was recognizable. Then other, lower S/N, events 
were examined for telltale SN signatures. If all the events 
had had S/N < 40, it is unlikely the SNe would have ever 
been recognized. For this reason, one may also wonder 
whether some of the other "low-quality" (S/N ^ 30!) 
events are due to some o ther u nanticipated astrophys- 
ical effect. lAlcock et al.l l|2000() were not able to prove 
that they were not. However, because of the strong tail 
of very high S/N events, they were able to show that 
their conclusions did not rest on these events that were 
open to question: they repeated their analysis, progres- 
sively increasing (S/N)inin, and showed that their conclu- 
sions did not change (although of course their statistical 
errors then deteriorated). The high S/N tail therefore 
gave credence to lower S/N events, at least statistically, 
and these then allowed MACHO to better characterize 
the population they were detecting. 

Also shown in Figure [H is a (non-exhaustive) sa mple 
of RV planet detections from lCumming et alJ l)1999D and 



iVoet et al] l)2002f) . Note that the great majority of these 
have very high S/N. This partly reflects the extreme con- 
servatism of the observers. This conservatism is prac- 
ticed by the majority of the RV observational commu- 
nity and is reflected in the fact that, while the typi- 
cally quoted single-measurement RV error is only a few 
m s~^, very few planets have been announced with ve- 
locity semi-amplitude < 20 m s"-'^. It is just such conser- 
vatism that has established the RV planets as bedrock 
facts rather than hopeful speculation. The RV detec- 
tions of planets around HD219542b and e Eri are the 
exceptions that prove the rule. The tentative detection 
of a planet around Hp 219542b was originally claimed by 
iDesidera et"al] lj2003j) . This detection was at S/N - 8, 
near the S/N thre shold pro posed for terrestrial planet 
searches. However. iDesidera e t al. (2Q^ cautioned that 
the R V variations could also be due to stellar activity. In- 
deed, jOeside^^On <|2004() recently reported additional 
measurements indicating that the RV variations are in- 
deed most likely due to stellar activity. The RV detec- 
tion of a p lanet around e Eri was at an even higher 
S/N ~ 13 l|Hatzes et al.ll2000() . However, despite this 
relatively high S/N (by the standards of proposed ter- 
restrial planet searches) and despite the strong exter- 
nal prior favoring a planet around e Eri ijNimovl 119751 
1995), this detection remains controversial. Also very in- 
structive is the extremel y strong (S/N ^ 4 0) detection 
reported for HD192263 ISantos et a'Lll2000|) . which was 
later refiited l|Henrv et~al. 2002) and then subsequently 
reasserted (|Santos et al.ll2003 '). All three of these exam- 
ples show that extreme caution is warranted when prob- 
ing for extreme or novel phenomena. 

The microlensing-like SNe, HD219542b, and e Eri 
demonstrate that it is difficult to break new scientific 
ground with detections that, while formally robust, lack 
the S/N to unambiguously characterize the object that 
has been detected. On the contrary, the significance of 
such ambiguous "detections" can only be evaluated pro- 
vided that the sample includes other detections that are 
completely unambiguous. 

Now, it is true that experiments yielding null results 
can and do operate at o r near the pure-noise limit. For 
example. iGilliland et al.l l)2000l) was able to set limits on 
planets in 47 Tuc going down to a threshold (S/N)inin — 
6.3 at which point they expected 0{1) false detection 
due to Gaussian noise. Gaudi et al. (2002) searched for 
planetary signatures in 43 microlensing events down to 
(S/N)niin = 7.7, a factor 1.5 higher than the thresh- 
old dictated by Gaussian noise. Their retreat from this 
threshold is highly instructive. They found about a half 
dozen events above the Gaussian threshold but below 
their adopted one. They concluded that these could not 
be regarded as reliable planet detections partly because 
a Monte Carlo analysis of stable light curves showed a 
similar S/N distribution, and partly because of the ex- 
pected form of the microlensing S/N distribution shown 
in Figure That is, if a substantial fraction of these 
marginal "detections" were real, one would also expect 
other detections at much higher S/N. This again demon- 
strates that without good sensitivity to high S/N detec- 
tions, a cutting-edge experiment cannot reliably interpret 
the marginal detections. 

6. RESULTS 
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6.1. Number of Planets at Realistic Thresholds 

Considering all the arguments presented in ^ we con- 
clude that, although thresholds of (S/N)min ^ 8 may 
yield formally statistically significant detections, conser- 
vative thresholds of (S/N)min ~ 25 should be adopted 
for the ambitious, cutting-edge experiments discussed 
here. In light of this conclusion, it is instructive to 
reconsider the signal-to-noise scalings show in Figure ^ 
and discussed in 21 Table 1 summarizes the number of 
expected Earth-mass planet detections for various val- 
ues of (S/N)min. Several important points are worth 
mentioning. First, although transits (Kepler), astrome- 
try (SIM), and microlensing all have similar sensitivities 
when compared at a minimal requirement (S/N)njiii = 8, 
the sensitivities of both SIM and Kepler fall off very 
steeply compared to microlensing (N oc (S/N)"?;'* versus 
A'" oc (S/N)~;j^). As a result, at the conservative thresh- 
old of (S/N)inin — 25, microlensing is 5 times more sensi- 
tive than either transits or astrometry. Furthermore, for 
the specific assumptions (see fJ^J adopted here, the total 
number of Earth-mass planets detected by Kepler and 
SIM above this threshold is relatively small, < 10. In 
the case of SIM, this may be improved by more judicious 
appropriation of the observing time f i)4.1|l . and the sit- 
uation will be better for shorter periods (for Kepler) or 
longer periods (for SIM). However, it is generally the case 
that the assumptions we have made are optimistic, as 
we have ignored known (and unanticipated) effects that 
will degrade the sensitivity, assumed that all stars have 
Earth-mass planets, and assumed observational strate- 
gies that are more aggressive than those currently being 
planned for these experiments. Therefore, it seems likely 
that, when more realistic assumptions are adopted, the 
number of expected robust detection of Earth- mass plan- 
ets will be modest. 

6.2. Sensitivity as a Function of Stellar Mass 

What are the masses of the stars whose planets would 
be detected in the four surveys summarized in Figure^ 
Figure 01 shows these distributions for two different S/N 
thresholds, (S/N),„i„ = 8 and (S/N),„i„ = 25. 

At (S/N)niin = 8, which is similar to the thresholds 
that are customarily discussed in this field, microlensing, 
astrometry, and transits all have rather similar distri- 
butions, with microlensing extending to somewhat lower 
masses and transits to somewhat higher masses. These 
broad distributions primarily reflect the breadth of the 
underlying stellar mass function. The if epter experiment 
tends to be cut off at low masses because these faint stars 
fall below the sky of its 10" PSF (Gould ct al. 2003b). 
Microlensing is cut off at high masses because there are 
no such stars in the Galactic bulge, which is the location 
of most of the lens population. RV is especially sen- 
sitive to high-mass stars because these are bright and, 
except for stars V < 1.5, our assumed sensitivity scales 
oc lO""'^^. In fact, the most massive stars in the RV 
histogram are A stars and therefore probably inaccessi- 
ble to RV. This subtlety has been ignored in our simple 
prescription for detectability based on V flux alone. 

However, as we discussed in § El historical experience 
argues against the viability and believability of S/N — 8 
detections in challenging experiments of this type. Based 
on this historical experience, we believe that the experi- 
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Fig. 3. — Distributions of parent-star masses for the exper- 
iments summarized in Fig. for two different S/N thresholds, 
(S/N)jnin = 8 and (S/N)niin = 25. At (S/N)min = 8, microlensing, 
transits (Kepler), and astrometry [SIM) all have comparably broad 
parent-star mass distributions with comparable normalizations. At 
(S/N)niin = 25, the forms of the distributions remain similar, but 
microlensing has a much higher normalization than the other two. 



mental design should hinge on the expected number and 
distribution of detections at much higher S/N, perhaps 
(S/N)min = 25. This is also shown in Figure El The 
functional forms of the distributions are similar to the 
case of (S/N)niin = 8, but the amplitudes are reduced. 
As expected from Figure Hand the scaling relations dis- 
cussed in §^ microlensing drops by about a factor of 3, 
while astrometry and transits drop by about a factor 15. 
RV would not detect Earth-mass planets at all at this 
threshold. 

6.3. Sensitivity as a Function of Planet Period 

As discussed in § 13 our primary comparison of differ- 
ent techniques has allowed each to play to its strengths. 
While democratic, this approach also means that Fig- 
ures EEl do not reflect the relative detectability of the 
same ensemble of planetary systems. To better under- 
stand the complementarity of different techniques that 
was discussed in § |21 we plot in Figure ^ the number of 
detectable planets as a function of period, P. In this case, 
we assume that each star has one planet at each period. 
As expected, transits and RV are primarily sensitive to 
short-period planets, while microlensing and astrometry 
are primarily sensitive to long-period planets. 

6.4. Sensitivity as a Function of Planet Mass 

For the majority of the discussion, we have concen- 
trated on planets with M = (or R = i?©), since 
all the currently-known terrestrial planets have a mass 
< Mq. However, some theories of planet formation pre- 
dict that considerably more massive te rrestrial planets 
with M S. 10 M(T, may be common (e.e..\lda fc Lin 20041 
IKokubo fc idaHiionRavmond et al.il2004D . Therefore 
we briefly consider the number of expected planet detec- 
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Fig. 4. — Sensitivity to planets as a function of period P for the 
four techniques. In contrast to Figs ITIIsl here it is assumed that 
every star has a planet with period P, so that each technique is 
confronted with the same ensemble of planetary systems. Transits 
{Kepler) are truncated at P = 2 yr because Kepler is a four-year 
experiment that must detect at least 2 transits. Astrometry {SIM) 
has a dotted-line extension from P = 5 yr to P = 10 yr to reflect the 
possible extension of this mission from 5 to 10 years. The survey 
designs are identical to those described in § |2] and the indicated 
S/N thresholds are the same as in Fig. O 



tions as a function of the mass of the planet. For both 
RV and astrometry, the signal is directly proportional to 
the mass of the planet. When combined with the S/N 
scalings from [Ql this yields N oc M^ *^ for astrometry 
and N oc M"^ -"^ for RV. For Kepler, the signal is oc i?^, 
or, assuming constant planet densities, oc M"^/^ . There- 
fore, N oc M^-^. Finally, for microlensing, the signal is 
oc M^/^. However, here we must also consider the in- 
crease in the detection probability for the larger mass 
ratios, which is oc sources below 

the turnoff and fainter than the background, N oc M^^^. 
We confirm this scaling in our simulations. 

7. DISCUSSION 

As discussed in § 13 and demonstrated in Figures ^ 
and ^ the four detection techniques reviewed here are 
very distinct, not only in terms of their sensitivity as 
a function of S/N and period, but also in the informa- 
tion they yield about the planets. In addition, there are 
number of secondary characteristics of the planet sam- 
ple gathered by the experiments that are unique to each 
of the four methods. These characteristics may also be 
important to consider when assessing how well-suited a 
given method is to fulfilling a particular task. In the 
following, we examine the relative strengths and weak- 
ness of each method, considering both the primary and 
secondary characteristics of the planet harvest, and con- 
centrating on five interrelated considerations: the total 
number of planets detected, the sensitivity to habitable 
planets, the ability to robustly calibrate the frequency 
of terrestrial planets as required for the success of TPF, 
the location and nature of the planet-bearing stars, and 



the opportunity for follow-up of the detected planets and 
hosts. 

The RV planet search envisioned here is clearly the 
least competitive in terms of the total number of plan- 
ets detected. Even an optimized search using 4 dedi- 
cated 2m telescopes searching for 5 years would only al- 
low the detection (at S/N = 25) of Earth-mass planets 
with P = 0.5 yr around the five best candidate host stars. 
The sensitivity is further reduced for planets in the hab- 
itable zone. On the other hand, the technology required 
to initiate such a search is already available, and the de- 
tection of even one such planet would be very interesting, 
because the host stars would be very nearby d <2Q pc, 
and thus promising targets for detailed follow-up with 
ground-based instruments, as well as TPF. Furthermore, 
the fact that such a search is ground-based and targeted 
means that it is very flexible. Therefore, it may be possi- 
ble to search for Earth-mass planets around a larger num- 
ber of stars using a two-tier survey approach, in which 
promising targets from an initial, low-S/N survey are fol- 
lowed up more intensively to yield high-S/N detections. 
Finally, RV searches are advantageous in that they can 
detect multi-planet systems, with both gas-giant and ter- 
restrial planets, providing essential constraints on the ef- 
fects of massive planets on terrestrial planet formation. 
Furthermore, planets detected via RV can be confirmed 
by SIM, yielding planet masses. 

In 'survey' mode, in which all targets are allocated 
the same amount of observing time, and for conserva- 
tive thresholds of (S/N) = 25, 5/M will detect at most 
~ 10 Earth-mass planets at P = 3 yr. For the largest 
detectable period corresponding to the nominal mission 
lifetime of P = 5 yr, ~ 30 Earth-mass planets are de- 
tectable. However, it may be possible to detect as many 
as ~ 80 Earth-mass planets at S/N = 25 and P = 3 yr 
if the search is optimized, and the observing time is al- 
located according to the sensitivity of the host star to 
planets (see i)4.1|l . As discussed above in the context 
of RV searches, it may be possible to survey an even 
larger number of stars for Earth-mass planets by adopt- 
ing a dynamic, two-tier approach. SIM will also be 
sensitive to planets in the h abitable zone of some tar- 
get stars ijSozzetti et al.ll2?)02»l , although maximizing the 
number of detections of such pl anets requires a mo difi- 
cation of the observing strategy ijGould et alJl2003'ah . In 
addition, the stars surveyed by SIM are exactly those 
which can be targeted by TPF, making the results of 
the SIM planet search critical. However, we emphasize 
that the results presented here assume a very ambitious 
SIM planet search. In particular, we have assumed that 
a total of ~ 10, 000 hours, or ~ 20% of the available 
SIM time, is devoted to searching for planets. Currently 
only ~ 8% of SIM time is allocated to searching for 
planets around nearby main-sequence stars, in two SIM 
key projects. Furthermore, the two key projects are not 
currently optimized to find Earth-mass planets. There- 
fore, achieving the results presented here will require not 
only an increase in the amount of time alloted to planet 
searches, but also a careful reassessment of the observing 
strategies. 

The primary advantage of Kepler is that it will be 
sensitive to Earth-mass planets in the habitable zone. 
Unfortunately, at conservative thresholds of S/N ^ 25, 
Kepler will only find ^ 6 Earth-mass habitable planets. 
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Although the detection of even one such planet would 
be extremely interesting, this is an insufScient number 
to accurately constrain the frequency of such planets. 
Therefore, constraints on this frequency will have to be 
derived from lower S/N observations, or from extrapo- 
lations from the frequency of higher-mass planets. Both 
of these procedures are potentially risky. Kepler is much 
more sensitive to planets with shorter periods (see Figure 
0)), and should provide robust statistics on terrestrial- 
mass planets close to their parent stars. Follow-up of 
candidate Earth-mass planets detected by Kepler will 
be difficult. The target stars of Kepler are disk main- 
sequence stars at distances of ~ kpc, and thus will be 
too faint {V > 9) or distant for confirmation with either 
RV or SIM. Naively, one might expect the Hubble Space 
Telescope (HST), provided that it is still operational, to 
yield S/N that are larger by a factor of two, given that its 
aperture is twice that of Kepler. However, this is more 
than compensated for by the fact that Kepler's bandpass 
is larger by a factor of ~ 10 than the 50 nm bandpass 
used to observe the transiting planet HD209458b, which 
iBrown et alJ l)200lD found to yield photon rates that are 
a factor of ~ 5 smaller than those expected for Kepler for 
stars of the same magnitude. Therefore, one expects S/N 
that are a factor of ^ 2 smaller for HST. It may be pos- 
sible to follow-up candidate transit events in the infrared 
using the James Webb Space Telescope {JWST). This 
will depend on the photometric precision of the avail- 
able JWST instruments and on whether it is launched in 
time to confirm the Kepler transits before their phases 
are lost. 

Of the four methods we have discussed, microlensing 
is the most sensitive in terms of the total number of po- 
tential planet detections at realistic S/N thresholds. At 
S/N ^ 25, a 5-year ground-based microlensing survey 
of the kind envisioned here could potentially detect ~ 50 
Earth-mass planets at a = 2.5 AU. Therefore, microlens- 
ing could constrain the frequency of terrestrial planets to 
reasonable precision. Although such a survey would also 
be sensitive to Earth-mass planets at lAU, this is only 
for low-mass target stars, and thus microlensing is not 
able to directly constrain the frequency of Earth-mass 
planets in the habitable zone. While a ground-based mi- 
crolensing survey is attractive in that it is flexible, rela- 
tively inexpensive, and does not rely on untested technol- 
ogy, the results stated above are predicated on the ability 
to achie ve near pho ton-limited photometry on stars with 
/ > 20 l|Gaudi et al.. 2004') . This may be diflicult in the 
crowded fields toward the Galactic bulge ( Bennett 2004). 
A space-based microlensing survey such as MPF would 
circumvent the problems with crowded-field photome- 
try. Detailed simulations of a space-based microlensing 
survey indicate that it would yield a detection rate of 
Earth-mass planets at ~ 2.5AU that is higher than the 
groun d-based survey by at least ~ 30% ([Bennett fc Rhiel 
1200^ . Further more, the space-based survey would be 
sensitive to planets over a wider range of semi-major 
axes, and would be considerably more sensitive to multi- 
ple planets. Both ground- and spaced-based microlensing 
surveys are sensitive to Earth-mass planets surrounding 
main-sequence stars with distances of 1 — 10 kpc. It 
is important to emphasize that microlensing is sensitive 



to planets around both Galactic b ulge and d isk stars. 
For example, in the simulations of iGaudi et a l. ( 2004), 
~ 30% of the detected Earth-mass planets orbit disk 
main-sequence stars. Because of the large distances to 
the lens stars, follow-up of detected planet candidates 
will be difficult. However, in many cases the lens star 
should be sufficiently bright to be detectable, and it will 
be possible to constrain the mass of the primary, and 
thus the mass of the planet (Bennett & Rhic 2002). Fur- 
thermore, by combining ground and space-based surveys, 
it should be possib le to routinely measure planet masses 
l)Gould et al.ll200^ . However, direct confirmation of the 
planet candidates via, e.g. RV or astrometric follow-up, 
will be essentially impossible in the foreseeable future. 

The synergy of the four methods for detecting Earth- 
mass planets is clear from Figure 0] which demonstrates 
the possibility of searching for Earth-mass planets over 
many decades in period by combining a variety of tech- 
niques. Since nothing whatsoever is known about the 
distribution or frequency of Earth-mass planets around 
other stars, no technique can be guaranteed to be the 
most sensitive. Any technique could yield null results if 
the regions of its greatest sensitivity were sparsely pop- 
ulated by planets. Therefore, a combined attack on the 
problem using all four techniques is indicated. The dis- 
cussions above indicate that the result of such a coor- 
dinated search for planets could go a long way toward 
answering many of the questions of interest, and would 
provide important calibrations for the next generation 
of experiments, including direct searches like TPF and 
Darwin. 

That said, the results presented here, and in particu- 
lar as shown in Figure 0] also reveal some rather sober- 
ing realities. The combined sensitivity of the four tech- 
niques reaches its nadir at P ~ 1 yr, near the peri- 
ods of the only two Earth-mass planets known to be 
orbiting a main-sequence star. Furthermore, at the 
(S/N)„iin = 25 threshold that historical experience sug- 
gests is the minimum for reliable results on such cutting- 
edge experiments, the number of expected detections of 
Earth-mass planets with P ~ 1 yr is frighteningly low. 
Even if all stars had such planets and even if all four 
of these techniques were funded and/or allocated tele- 
scope time at the level we have somewhat optimistically 
supposed, and even if all worked according to their spec- 
ifications, only about 5 such planets would be detected 
at (S/N)niin — 25. Indeed, unless Earth-mass planets 
are very common, or unless they are packed much closer 
to their parent stars than is true in the solar system, 
then it is possible that neither Kepler nor SIM (as they 
are currently planned) would yield any reliable, high-S/N 
Earth-mass planet detections. 
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